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Abstract. A detailed description of topology of integrable billiard systems 
is given. For elliptical billiards and geodesic billiards on ellipsoid, the corre- 
sponding Fomenko graphs are constructed. 
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1. Introduction 

An important and geometrically significant class of dynamical systems consists 
of billiards with ellipsoidal boundary in E d . By Chasles theorem, any line in the d- 
dimensional space is tangent to exactly d—1 quadrics from a given confocal family. 
These d—1 caustics, confocal to the boundary of the billiard, are fixed for each 
segment of a given trajectory [2]. Existence of caustics is, in fact, a geometrical 
manifestation of integrals of motion. Due to them, we easily see that elliptical 
billiard is an completely integrable Hamiltonian system. 

Famous classical example of integrable systems is geodesic flow on an ellipsoid 
£ in E d [19J . This system is closely related with billiards. In one way, the geodesic 
flow tends to the billiard flow inside an ellipsoid in E d_1 when the smallest axis 
of £ tends to zero. On the other hand, the geodesic flow on £ is a limit of the 
billiard motion inside £, when one of the caustics is a confocal ellipsoid tending to 
the boundary. Geodesic flow on ellipsoid also gives rise to another interesting class 
of billiard systems: billiards on the ellipsoid, where the boundary is determined 
by the intersection of the ellipsoid with a confocal quadric. Trajectories of such a 
billiard are composed of geodesic segments. 

Billiard systems within and geodesic flows on ellipsoids have been extensively 
studied from XlXth century on: see classical works [10l [13l [19l [23l [26] and more 
recent ones [ip[Ti T IT2 TIT5W^P D | IM P 5 P 7 | IM] with the references therein. From the 
non-vanishing interest, one may notice that this topic is far from being exhausted. 

l 
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The object of this paper is to give topological description of such systems using 
Fomenko graphs. The detailed description of this kind of topological classification 
of integrable systems can be found in [3-[9] and references therein, while a very 
concise summary for a reader not acquainted with it is given in the appendix of 
this paper. Although plane elliptical billiards are quite well known, we may see 
that such a description will give a new and exciting insight into its properties - 
note, for example, appearance of a non-orientable periodic trajectory along one of 
the axes of the billiard boundary in Proposition [31 see Figure @] 

In the book [7] , one may find a large list of Fomenko graphs for known integrable 
systems, such as integrable cases of rigid body motion and integrable geodesic 
flows on surfaces. This paper provides an additional set of such examples. Some 
other examples, connected with near-integrable dynamics can be found in |29j while 
bifurcations of Liouville foliations in a certain class of integrable systems with two 
degrees of freedom, were classified using Fomenko graphs in [28] . 

Let us note that topological properties of elliptical billiards, viewed as discrete 
dynamical systems, have been recently studied in [31 . However, in this paper, we 
consider billiards as continuous systems, which enables us to use tools developed 
by Fomenko and his school. 

This article is organized as follows. In Section [2j we begin with introductory 
remarks on isoenergy surfaces of billiard systems. Section [3] contains topological 
description of several examples of plane billiards, with the boundary composed of 
confocal conies, together with their representation via Fomenko graphs. In Section 
[H topological description of the billiard flow with the quadratic boundary on the 
ellipsoid in E 3 is given. We conclude this section by demonstrating the Liouville 
equivalence of the billiard motion within an ellipsoid on any Liouville surface. In 
Section^ the topological structure of the billiard inside ellipsoid in E 3 is described. 
The Appendix contains a summary on Fomenko graphs. 

2. Isoenergy Surfaces of Billiard Systems 

Let M n be an n-dimensional Riemann manifold, and f2 C M. a domain with the 
boundary composed of several smooth hypersurfaces. The billiard [22] inside Vt is a 
dynamical system where a material point of the unit mass is freely moving inside the 
domain and obeying the reflection law at the boundary, i.e. having congruent impact 
and reflection angles with the space tangent to the boundary at any bouncing point. 
It is also assumed that the reflection is absolutely elastic, i.e. that the speed of the 
material point does not change before and after impacts. 

It is important to remark that the change of the total energy of the system, only 
the intensity of the velocity vector of the point will be changed while the trajectories 
will be the same at each energy level. That is why, for the complete analysis of the 
billiard system, we may fix the speed and investigate only one energy level. 

The isoenergy space for the billiard system inside CI is: 

B = { (x,v) \ xeSl, veT x M, H = i}/~, 
{x, u) ~ (x, v) ^ x £ dfl and u — v _L T x dfl. 

Although the smoothness of motion of the billiard particle is violated at the 
boundary, let us note that, unexpectedly, the isoenergy space will be smooth, as- 
suming that the billiard boundary is smooth. Even more, this manifold is smooth 
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also when boundary is only composed of a few smooth parts, if the billiard reflection 
can be continuously defined at the angle points. This will be always the case in 
our examples, when the boundary is placed on a few confocal conies, since they are 
orthogonal to each other. Thus, the reflection at the intersection points is defined 
as follows: the velocity vector v after the reflection at the non-smooth point of the 
boundary, changes to —v. 



3. Plane Elliptical Billiards 

An important example of integrable billiard system is the billiard within an 
ellipse in E 2 . The integrability of such a billiard motion is due to the nice and 
elementary fact: each segment of a trajectory is tangent to the same conic confocal 
with the boundary [2j[4]. Moreover, the boundary of any integrable billiard in a 
plane domain is composed of segments of several confocal conies [5] . 

In this section, we will describe the topology of the plane billiards with the 
elliptic boundary. We will use the Fomenko graphs [8] to represent the isoenergy 
surfaces. 

Suppose that the ellipse in the plane is given by: 

2 2 

£ : — + 4- = 1, a>b>0. 
a b 

The family of conies confocal with £ is: 

2 2 
X V 

: +-«_ = 1, /i6R. 

a — fi o — /i 

We are going to consider billiard systems inside a bounded domain f2 in E 2 , 
whose boundary is a union of arcs of several confocal conies. As in Introduction, 
we denote its isoenergy manifold by B. 

Proposition 1. The isoenergy manifold corresponding to the billiard system within 
an ellipse in E 2 is represented by the Fomenko graph on Figured 




Figure 1 . Fomenko graph corresponding to the billiard within an ellipse 



The rotation functions corresponding to the edges of the graph are monotonous, 
and the limits of these functions on the lower edges are egual to oo approaching to 
A-atom and 2 approaching to B-atom; on the upper edge the limit is oo approaching 
to A-atom and 1 approaching to H-atom. 
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Proof. The isoenergy manifold B is the solid torus with the identification on ~ 
on the boundary. This identification glues pairs of points, while points contained 
on two curves - representing flows in positive and negative directions along the 
boundary of the billiard table, are not glued with any other. The torus with the 
two curves on the boundary is shown on Figure [2j These two curves correspond 




FIGURE 2. The isoenergy manifold for the billiard system within 
an ellipse 

to the limit flow with the caustic Cq = £, and they are represented with the lower 
A-atoms on the graph. For the parameter < /i < b, i.e. when the caustic is an 
ellipse, we have two Liouville tori over each value of /i - one for each direction of 
rotation. 

Let us describe the level set \i = b. This level set contains exactly those billiard 
trajectories that pass through foci of the ellipse. Among them, there is one periodic 
trajectory - the motion along cc-axis, while others have the well-known property 
that their segments alternately pass through left and right focus of the ellipse. 
These trajectories can be naturally divided into 2 classes - one is composed of 
the trajectories where the particle is moving upward through the left focus and 
downward through the right one, and the second contains the trajectories with 
the reverse property. Note these two classes by Si and S2 respectively. All their 
trajectories are homoclinically tending to the x-axis, thus Si and 1S2 are separatrices. 
Note that the periodic trajectory is orientable, so this level set corresponds to the 
B-atom. 

Note that Si (resp. S2) is the limit set of the family of Liouville tori corresponding 
to the flow with elliptic caustic in the clockwise (resp. counterclockwise) direction. 

When b < \i < a, i.e. when the caustic is a hyperbola, there is only one torus for 
each value. Finally, to the fx = a, the periodic motion along y-axis takes place, and 
it is represented by the upper A-atom. □ 

In the similar way, we can obtain the following: 

Proposition 2. The isoenergy manifold corresponding to the billiard system in the 
domain limited with two confocal ellipses in E 2 is represented by the Fomenko graph 
on Figured 

Proof. As in Proposition [TJ the lower A-atoms correspond to the limit flows on the 
boundary of the outer ellipse. 
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Figure 3. Billiard between two confocal ellipses and its Fomenko graph 



The level set for the caustic /i = b contains two periodic trajectories placed on 
the s-axis and all trajectories such that the continuations of their segments contain 
the foci. Let us note that such a trajectory is hcteroclinic and placed only in 
one of the half-planes with the ir-axis as the edge. Thus, this level set has four 
separatrices: two in the upper half-plane, two in the lower one. In each half-plane, 
one separatrix contains trajectories tending to the left periodic trajectory on the 
x-axis when time tends to oo and to the right one in — oo, and opposite for orbits 
on the other separatrix. Liouville tori corresponding to elliptic caustics tend when 
fj, tends to b, to two separatrices with the same movement direction - clockwise or 
counterclockwise. 

For hyperbolic caustics, the billiard table become split into two domains, sym- 
metric with respect to the cc-axis. Each family of Liouville tori tend to the two 
separatrices from the level /i = b that are contained in the same half-plane. 

This analysis shows that the atom that describes level set /i = b is C2. 

The upper A-atoms correspond to the periodic orbits along y-axis. □ 

Now we will consider the domain is determined by an ellipse and a confocal 
hyperbola. Let us note that the phase space in the cases, where the boundary 
is not smooth at every point, must be considered with a special attention. Since 
confocal conies are always orthogonal in the intersection points, the periodic flows 
along smooth arcs may appear as limits of the billiard motion, when the caustic 
tend to the conic containing the arc. Thus, although the tangent space to the 
boundary in the intersection points is not defined, we will take that the "allowed" 
velocity vectors in such points are those tangent to the curves containing these 
points, with the opposite vectors identified with each other. 

Proposition 3. Consider the billiard domain with the border composed of an ellipse 
and a confocal hyperbola. Then: 

1 If the domain is outside hyperbola, then the isoenergy manifold is repre- 
sented by the graph on Figure [^} 

2 If the domain is inside hyperbola, then the isoenergy manifold is represented 
by the graph on Figured 

Proof. In the first case, each of the A-atoms corresponds to the limit periodic orbit 
along one of the smooth arcs constituting the boundary of the billiard domain. The 
A*-atom represents the periodic orbit along x-axis and its homoclinic trajectories. 
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Figure 4. Billiard between ellipse and one branch of hyperbola 
and its Fomenko graph 




FIGURE 5. Billiard inside ellipse and hyperbola and its Fomenko graph 

In the second case, the lower A-atoms correspond to the limit motion on two 
arcs of the ellipse, the B-atom to the periodic orbit on x-axis and its homoclinic 
orbits, and the upper A atom to periodic orbit along y-axis. It is interesting to note 
that this system is Liouville equivalent to the billiard within the ellipse (compare 
Figures Q] and E|) . □ 

We conclude this section by analyzing an example when the border of the billiard 
table is continuously changed, to see how the bifurcations in the isoenergy manifolds 
appear. 

Proposition 4. Consider a billiard domain limited by two confocal ellipses and 
two arcs contained in different branches of a confocal hyperbola. 

1 If the domain is inside the hyperbola, then the isoenergy manifold is repre- 
sented by the graph on Figure^ 

2 If the x-axis is a part of the boundary, then the isoenergy manifold is repre- 
sented by the graph on Figure^ Note that V corresponds to a degenerated 
singular leaf containing two orientable periodic orbits and two heteroclinic 
separatrices. 

3 If the domain is as shown on the left side of Figure [51 then the isoenergy 
manifold is represented by the graph on the right side of Figure 

Remark 1. Near-integrable bifurcations of billiards from Propositions^ and^ can 
be studied by tools developed in [30] - 
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Figure 6. Billiard motion between two ellipses and hyperbola and 
the corresponding Fomenko graph 




Figure 7. Billiard motion between two ellipses and a degenerate 
hyperbola with the Fomenko graph 




Figure 8. Billiard motion between two ellipses and hyperbola and 
the corresponding Fomenko graph 

4. Billiards on Ellipsoids and Liouville surfaces 

In this section, we are going to analyze the topology of the billiard motion on the 
ellipsoid in E 3 , with the boundary cut by a confocal quadric surface. Next, we will 
consider the billiards within generalized ellipses on Liouville surfaces. Obtained 
results are going to be compared with those from the previous section. 

4.1. Topology of Geodesic Motion on Ellipsoid in E 3 . Since the segments of 
billiard trajectories on the ellipsoid are placed on geodesic lines, it is essential to 
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consider topology of the isoenergy surfaces for the geodesic motion on the ellipsoid. 
This topology is completely described in [6]. 

We suppose that the ellipsoid in E 3 is given by the equation: 

2 2 2 

X V z 

(1) £ : — + + — = 1, 0<c<b<a. 

a b c 

Theorem 1. [6,7\] The Fomenko graph for the Jacobi problem of geodesic lines 
on an ellipsoid is presented in Figured The rotation function that correspond to 




Figure 9. Fomenko graph for the Jacobi problem 



the lower and upper edges of the graph are: 

, . f a <S>(\,a)d\ f b <S>{\,a)d\ 

Piower(a) = -fa-rr: — TT7 (a€(c,6)), p up p CT (a) = r c a - ,. — rjr (ae(6,a)), 
J b *(A,a)dA J a $(A,a)dA 

where 

$(A,a) ^ 



y/-X(a - \)(b - A)(c - A)(a - A) 

It is interesting to remark that this theorem is used to prove that the Jacobi 
problem of geodesic lines on an ellipsoid and the Euler case of the rigid body 
motion are orbitally equivalent (see [HE]). 

4.2. Billiards on Ellipsoid in E 3 . A generalized ellipse on the surface of the 
ellipsoid |T]) is the intersection of this ellipsoid with a confocal hyperboloid. Let 
us note that such an intersection consists of two disjoint closed curves, which are 
curvature lines on the ellipsoid. This intersection divides the surface of the ellipsoid 
into three domains: two of them are simply connected and congruent to each other, 
while the third one, placed between them is 1-connected. 

More precisely, any surface confocal with the ellipsoid (pJJ is given by the equation 
of the form (see Figure [TP]) : 

(2) Qa : Q A = _?L + J!L + = i. 

a — A o — A c — A 

Let the Jacobi coordinates (Ai, A2, A3) related to this system of confocal quadric 
surfaces be ordered by the condition Ai > A2 > A3. 

Then the one-sheeted hyperboloid Qp, (c < (3 < b), cuts out the following three 
domains on the surface of £: = {A 2 > (3, z > 0}, fif = {A2 > (3, z < 0} and 
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Figure 10. Confocal quadrics 

fig = {A2 < 0}. The first two domains are symmetric with respect to the xy-pl&ne, 
and the third one is the ring placed between them on £, as shown on Figure fTTl 




Figure 11. Intersection of ellipsoid with one-sheeted hyperboloid 

Similarly, the two-sheeted hyperboloid Q 7 , (b < 7 < a), determines the domains 
S17 = {Ai < 7, x > 0}, fXJ = {Ai < 7, x < 0} and QJ = {X 1 > 7}, see Figure [121 




Figure 12. Intersection of ellipsoid with two-sheeted hyperboloid 
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Proposition 5. The isoenergy manifolds corresponding to the billiard systems 
within the domains fif and flj on £ is represented by the Fomenko graph on Figure 

m 

The rotation functions for the billiard within fi^, for the lower and upper edges 
of the Fomenko graph respectively, are: 



Slower 



(a) 



J?*(A,a) 



(ae((3,b)), p(L por (a) 



J>(A,a) 



(a G (6, a)). 



XTie rotation functions for the billiard within QJ , for the lower and upper 
of the Fomenko graph respectively, are: 



7 

Slower 



(a) 



/;*(A,a) 
$ is defined as in Theorem]]^ 



(a 6(6,7)), p^per (a) = 



/ 6 7 ^(A,a) 
/ c a *(A,a) 



(a e (c,6)). 



4.3. Billiard within an Ellipse on a Liouville surface. Now, let us state the 
main result of this section. 

Theorem 2. All billiard systems within ellipse on an arbitrary Liouville surface are 
Liouville eguivalent. The Fomenko graph corresponding to an isoenergy manifold 
of such a system is represented on Figure [7J 

Proof. Families of confocal ellipses and hyperbolas, together with corresponding bil- 
liard systems, were defined and described by Darboux [13] (for a recent account on 
this topic, see [T51I16] ). Since such billiard have all topological properties analogous 
to billiards within an ellipse in the Euclidean plane, the theorem is proved. □ 

5. Billiards inside Ellipsoid in E 3 

Here, we are going to analyze the topology of the billiard motion within an 
ellipsoid in E 3 . We suppose that the equation of the ellipsoid is |T]). 

Each trajectory of the billiard motion within £ has exactly two caustics from the 
confocal family Q\. Suppose these two surfaces are Q\ 1 and Q\ 2 , A2 < Ai- Then 
the bifurcation set of an isoenergy surface of the system is given on Figure [T3J 



c b a 

Figure 13. The bifurcation set for the billiard within an ellipsoid 
in E 3 



The bifurcations of the Liouville tori for this system were investigated in [31IT41 



El]. 
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Now, let us explain in detail how the behaviour of the billiard motion is connected 
to the diagram on Figure [T3l 

Over each point in the gray area on the diagram, there is one or two 3-dimensional 
Liouville tori. The marked edges will correspond to the degenerated level sets. 

Now, let us describe in detail the level sets lying over different points in the 
bifurcation set on the Figure [13j 



5.1. Regular Leaves. Over each point inside rectangles [c, b] x [0, c], [b,a] x [0, c] 
and the triangle with vertices (c, c), (b, c), (6,6), there are two Liouville tori T 3 . 
Over each point inside rectangle [b, a] x [c, b], there is only one T 3 . 

• Points inside [c, b] x [0, c] correspond to the motion with an ellipsoid and a 
one-sheeted hyperboloid as caustics. One Liouville torus is formed by the 
trajectories winding in the positive direction around the z-axis, and the 
other one by those winding in the negative direction. 

• Points inside [b, a] x [0, c] correspond to the case when one caustic is ellipsoid 
and the other a two-sheeted hyperboloid. Each Liouville torus is formed by 
the trajectories winding in one direction around the x-axis. 

• Points inside triangle (c, c) — (6, c) — (6, b) correspond to the motion with both 
caustics being different one-sheeted hyperboloids. Each of the Liouville tori 
is formed by the trajectories winding in one direction around the x-axis. 

• Points inside rectangle [b, a] x [c, b] correspond to the case when both caus- 
tics are hyperboloids, but of different type. Then, all corresponding billiard 
trajectories form one Liouville torus. 



5.2. Singular Leaves. Singular leaves are lying over edges of the diagram repre- 
sented on Figure [T51 They appear when one or both caustics are degenerated. 

First, let us clarify the notion of a degenerated quadric from confocal family (|2j). 
as well as the geometrical meaning of the billiard motion with such a caustic. 

The degenerated quadrics Q\, A 6 {a,6, c} are the following curve lying in the 
coordinate planes: 

y 2 z 2 

Qa ■ y —r =1, ar = 0, 

a — b a — c 

„ x 2 z 2 

Q b ■■ r-T — = 1, y = 0, 

a — b b — c 

1 2 
X V 

Qc : + r^ = 1 > z = - 

a — c b — c 

Notice that Q c is an ellipse in the icy-plane, Qb is a hyperbola in the xz-plane, 
while Q c is, in the real case, the empty set. Nevertheless, we will consider Q a as an 
abstract curve in the yz-plane. The degenerated quadrics are depicted on Figure 

m 

The billiard motion with some of these three degenerated caustics Q\ is cither 
such that each segment of the trajectory intersects the curve Qa or the whole 
trajectory is lying in the coordinate plane containing this curve. In the latter case, 
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Figure 14. Ellipsoid £ and degenerated quadrics Qb, Q c 
the motion reduces to the plane billiard one of the ellipses: 



y 2 
b 


z 2 
1 

c 


= i, 


x 2 


z 2 
V — 


= l, 


a 


c 


x 2 


y 2 


= l, 


a 





Notice that, in the case A = a, only plane trajectories exist. 

Each of these three ellipses is the intersection of £ with one of the coordinate 
hyper-planes (see Figure fT5|) . 




Figure 15. Ellipsoid £ and ellipses £ a , £b, £ c 

Besides these cases with a degenerated caustic, there one more special case - 
when one of the caustics is the boundary ellipsoid £ . This case can be considered 
as the limit case with the caustic Q\, A — > 0+, and its trajectories are geodesic 
lines on £. 

Now, let us analyze the edges of the bifurcation diagram on Figure IT51 

Edge [c, a] x {0}. As already mentioned, the motion corresponding to this edge is 
the geodesic motion on £ . Thus, the topology of the corresponding subset in the 
isoenergy manifold is completely described by the Fomenko graph on Figure [9] 
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Let us summarize this case: 

• Point (c, 0) corresponds to the motion along the curve £ c . There are two 
1-dimcnsional tori T 1 lying over the point (c, 0) of the bifurcation diagram 
- each one corresponding to the flow in one direction. 

• Inner points of the segment [c, b] on Ai-axis correspond to the geodesic mo- 
tion on £ with a one-sheeted hyperboloid as caustic. Such geodesic lines fill 
the ring between two intersection curves of £ and the hyperboloid. Over 
each point inside the segment, there are two tori T 2 - each one correspond- 
ing to winding in one direction around the z-axis. 

• The level set corresponding to (6, 0) contains two periodic trajectories and 
four two-dimensional separatrices. The periodic trajectories are placed 
along the curve £b- Trajectories on the separatrices are passing through 
the umbilical points of the ellipsoid. 

• Inner points of the segment [b, a] on Ai-axis correspond to the geodesic mo- 
tion on £ with a two-sheeted hyperboloid as caustic. Such geodesic lines fill 
the ring between two intersection curves of £ and the hyperboloid. Over 
each point inside the segment, there are two tori T 2 - each one correspond- 
ing to winding in one direction around the x-axis. 

• Point (a, 0) corresponds to the motion along £ a . There are two 1-dimensional 
tori T 1 lying over this point - each one corresponding to the motion in one 
direction. 

Edge {a} x [0, b] . This edge corresponds to the billiard motion with the degenerated 
caustic Q a - All trajectories of such a motion are placed in the yz-plane, thus this 
edge in fact corresponds to the billiard within an ellipse. The topology of the set 
lying over this edge is completely described by the Fomenko atom on Figure [1] 

Edge [b, a] x {&}. This edge corresponds to the billiard motion with one degenerate 
caustic Qb, and the other caustic being a two-sheeted hyperboloid. 

Acknowledgements. We thank A. V. Bolsinov and V. Rom-Kedar for interesting 
discussions. This work is partially supported by the Serbian Ministry of Science 
and Technology, Project no. 144014: Geometry and Topology of Manifolds and 
Integrable Dynamical Systems. 

Appendix: Fomenko graphs 

In the appendix, we are going to give a concise description of the representation 
of isoenergy surfaces and some of their topological invariants by Fomenko graphs. 
For all details, see [6 , 9J and references therein. 

Let (A4, tu, H, K) be an integrable system given on a four-dimensional mani- 
fold with a symplectic form u>. Here H, K denote independent functions on A4, 
commuting with respect to the symplectic structure on A4. 

Level sets are subsets of M. given by equations H = const, K = const. The 
Liouville foliation of M. is its decomposition into connected components of the 
level sets. A leave of the foliations is regular if dH, dK are independent at each 
its point, otherwise it is called singular. Singular leaves satisfying some additional 
conditions will be called non- degenerate. For these non-degeneracy conditions, see 
i- 
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We suppose that isoenergy surface H — const are compact, thus each foliation 
leaf will also be compact. Thus, by the Arnol'd-Liouville theorem [2], each regular 
leaf is diffeomorphic to the 2-dimensional torus T 2 and the the motion in its neigh- 
borhood is completely described by, for example, the action-angle coordinates. We 
say that isoenergy surfaces are Liouville equivalent if they are topologically conju- 
gate and the homeomorphism preserves their Liouville foliations 

The set of topological invariants that describe completely isoenergy manifolds 
containing regular and non-degenerate leaves not containing fixed points of , up to 
their Liouville equivalence, consists of: 

• The oriented graph G, whose vertices correspond to the singular connected 
components of the level sets of K, and edges to one-parameter families of 
Liouville tori; 

• The collection of Fomenko atoms, such that each atom marks exactly one 
vertex of the graph G; 

• The collection of pairs of numbers (rj,£j), with G ([0,1) H Q) U {oo}, 
£i & {—1,1}, 1 < i < n. Here, n is the number of edges of the graph G and 
each pair (rj,£j) marks an edge of the graph; 

• The collection of integers n\, ri2, ■ ■ ■ , n s . The numbers correspond to 
certain connected components of the subgraph G° of G. G° consists of all 
vertices of G and the edges marked with = oo. The connected com- 
ponents marked with integers are those that do not contain a vertex 
corresponding to an isolated critical circle (an A atom) on the manifold Q. 

Let us clarify the meaning of these invariants. 

First, we are going to describe the construction of the graph G from the manifold 
Q. Each singular leaf of the Liouville foliation corresponds to exactly one vertex 
of the graph. If we cut Q along such leaves, the manifold will fall apart into 
connected sets, each one consisting of one-parameter family of Liouville tori. Each 
of these families is represented by an edge of the graph G. The vertex of G which 
corresponds to a singular leaf C is incident to the edge corresponding to the family 
T of tori if and only if <9T n C is non empty. 

Note that dT has two connected components, each corresponding to a singular 
leaf. If the two singular leaves coincide, then the edge creates a loop connecting 
one vertex to itself. 

Now, when the graph is constructed, one need to add the orientation to each 
edge. This may be done arbitrarily, but, once determined, the orientation must 
stay fixed because the values of numerical Fomenko invariants depend on it. 

The Fomenko atom which corresponds to a singular leaf £ of a singular level set 
is determined by the topological type of the set C e . The set C e D C is the connected 
component of { p 6 Q | c — e < K(p) < c + e }, where c = K{C), and e > is such 
that c is the only critical value of the function K on Q in interval (c — s, c + e). 

Let C+ = { p e C £ | K {p) > c },£-={ p e C £ K(p) < c }. Each of the 
sets Cf , £~ is a union of several connected components, each component being a 
one-parameter family of Liouville tori. Each of these families corresponds to the 
beginning of an edge of the graph G incident to the vertex corresponding to C. 

Let us say a few words on the topological structure of the set C. This set consists 
of at least one fixed point or closed one-dimensional orbit of the Poisson action 3> 
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on M. and several (possibly none) two-dimensional orbits of the action, which are 
called separatrices. 

The trajectories on each of these two-dimensional separatrices is homoclinically 
or heteroclinically tending to the lower-dimensional orbits. The Liouville tori of 
each of the families in C e \ C = Cf U Cj tend, as the integral K approaches c, to 
a closure of a subset of the separatrix set. 

Fomenko Atoms. Fomcnko and his school completely described and classified 
non-degenerate leaves that do not contain fixed points of the Hamiltonian H . 

If C is not an isolated critical circle, then a sufficiently small neighborhood of 
each 1-dimensional orbit in C is isomorphic to cither two cylinders intersecting 
along the base circle, and then the orbit is orientable, or to two Moebius bands 
intersecting each other along the joint base circle, then the 1-dimensional orbit is 
non- orientable. 

The number of closed one-dimensional orbits in C is called the complexity of the 
corresponding atom. 

Fomenko atoms of complexity 1. There are exactly three such atoms. 

The atom A. This atom corresponds to a normally elliptic singular circle, which is 
isolated on the isoenergy surface Q. A small neighborhood of such a circle in Q is 
diffeomorphic to a solid torus. One of the sets Cf , £~ is empty, the other one is 
connected. Thus only one edge of the graph G is incident with the vertex marked 
with the letter atom A. 

The atom B. In this case, C consists of one orientable normally hyperbolic circle 
and two 2-dimensional separatrices - it is diffeomorphic to a direct product of the 
circle S 1 and the plane curve given by the equation y 2 — x 2 — x 4 . Because of its 
shape, we will refer to this curve as the 'figure eight'. The set C e \C has 3 connected 
components, two of them being placed in Cf and one in C ~ , or vice versa. Let us 
fix that two of them arc in Cf . Each of these two families of Liouville tori limits as 
K approaches c to only one of the separatrices. The tori in C~ tend to the union 
of the separatrices. 

The atom A* . C consists of one non-orientablc hyperbolic circle and one 2-dimen- 
sional separatrix. It is homeomorphic to the smooth bundle over S 1 with the 'figure 
eight' as fiber and the structural group consisting of the identity mapping and the 
central symmetry of the 'figure eight'. Both £+ and C~ are 1-parameter families 
of Liouville tori, one limiting to the separatrix from outside the 'figure eight' and 
the other from the interior part of the 'figure eight'. 

Fomenko atoms of complexity 2. There are six such atoms. However, we give 
here the description only of those appearing in the examples of this paper, see 
Figures El El 

The atom C2- C consists of two orientable circles 71, 72 and four heteroclinic 2- 
dimensional separatrices Si, S2, £3, S4. Trajectories on Si, S3 are approaching 
71 as time tend to 00, and 72 as time tend to —00, while those placed on 1S2, S4 
approach 72 as time tend to 00, and 71 as time tend to —00. Each of the sets Cf, 
C~ contains two families of Liouville tori. As K approaches to c, the tori from one 
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family in C e deform to S\ U 1S2, and from the other one to S3 U S4. The tori from 
one family in £+ is deformed to S\ U Si, and from the other to S2 U S3. 

The atom Di. C consists of two orientable circles 71, 72 and four 2-dimensional 
separatrices Si, S2, S3, S4. Trajectories of Si, S2 homoclinically tend to 71, 72 
respectively. Trajectories on S3 are approaching 71 as time tend to 00, and 72 as 
time tend to —00, while those placed on S4, approach 72 as time tend to 00, and 
71 as time tend to —00. One of the sets £+, C~ contains three, and the other one 
family of Liouville tori. Lets say that £+ contains three families. As K approaches 
to c, these families deform to Si, S2 and S3 PI S4 respectively, while the family 
contained in C~ deform to the whole separatrix set. 



This concludes the complete list of all atoms appearing in this paper. 

Numerical Fomenko Invariants. Each edge of the Fomcnko graph corresponds 
to a one-parameter family of Liouville tori. Let us cut each of these families along 
one Liouville torus. The manifold Q will disintegrate into pieces, each corresponding 
to the singular level set, i.e. to a part of the Fomenko graph containing only one 
vertex and the initial segments of the edges incident to this vertex. To reconstruct 
Q from these pieces, we need to identify the corresponding boundary tori. This can 
be done in different ways. Thus, the basis of cycles is fixed in a certain canonical 
way on each boundary torus. Denote by 6^ the bases corresponding to the 
beginning and the end of an edge respectively. Denote by: 



the transformation matrix from b i 
Marks r^. They arc defined as: 



H 
to b+. 



Marks £j. They are: 



n = 



Si = 




if ft ± 0, 
if ft = 0. 



sign ft, if ft 7^0, 
signer, if ft = 0. 



Marks n^. To determine marks rife, we cut all the edges of the graphs having finite 
marks r,. In this way the graph will fall apart into a few connected parts. Consider 
only parts not containing A-atoms. To each edge of the graph having a vertex in 
a given connected part, we join the following integer: 



Oil 



if ej is has the initial vertex in the connected part 



if a is has the ending vertex in the connected part 



if both verteces of e, are in the connected part. 
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Then to the connected part of the graph, we join the mark: 

Rotation Functions. Consider an arbitrary edge of a given Fomenko graph and 
recall that it represents a one-parameter family of Liouvillc tori. Let us choose one 
such a torus and fix on it a basis (A, fi) of cycles. By Arnol'd-Liouville theorem, the 
motion on the torus is linear, thus there are coordinates ((^1,(^2) S S 1 x S 1 such 
that the Hamiltonian vector field can be written in the form: 

d , d 
a x h6 « — ' 

Otpi Olfi2 

while the coordinate lines ip2 = const, ipi — const are equivalent to basic cycles A, 
[i. Rotation number on the torus is p = — . If we continuously extend the basis 

(A, n) to the other tori of the family, the collection of obtained rotation numbers 
will represent rotation function. 

References 

[I] Simonetta Abenda and Yuri Fedorov, Closed geodesies and billiards on quadrics related to 
elliptic KdV solutions, Letters in Mathematical Physics 76 (2006), 111-134. 

[2] Vladimir Arnold, Mathematical Methods of Classical Mechanics, Springer Verlag, New York, 
1978. 

[3] Michele Audin, Courbes algebriques et systemes integrables: geodesiques des quadriques, 

Expo. Math. 12 (1994), 193-226. 
[4] Marcel Berger, Geometry, Springer Verlag, Berlin, 1987. 

[5] S. V. Bolotin, Integrable Birkhoff billiards, Vestnik Moskov. Univ. Scr. I Mat. Mekh. (1990), 
no. 2, 33-36. 

[6] A. V. Bolsinov and A. T. Fomenko, The geodesic flow of an ellipsoid is orbitally equivalent 
to the Euler integrable case in the dynamics of a rigid body, Dokl. Akad. Nauk SSSR 339 
(1994), no. 3, 293-296. 

[7] , Integrable Hamiltonian Systems: Geometry, Topology, Classification, Chapman and 

Hall/CRC, Boca Roton, Florida, 2004. 

[8] A. V. Bolsinov, S. V. Matveev, and A. T. Fomenko, Topological classification of integrable 
Hamiltonian systems with two degrees of freedom. List of systems with small complexity, 
Russian Math. Surveys 45 (1990), no. 2, 59-94. 

[9] A. V. Bolsinov and A. A. Oshcmkov, Singularities of integrable Hamiltonian systems, Topo- 
logical Methods in the Theory of Integrable Systems, Cambridge Scientific Publ., 2006, pp. 1- 
67. 

[10] Arthur Cayley, Developments on the porism of the in- and- circumscribed polygon, Philosoph- 
ical magazine 7 (1854), 339-345. 

[II] S.-J. Chang, Bruno Crespi, and K.-J. Shi, Elliptical billiard systems and the full Poncelet's 
theorem in n dimensions, J. Math. Phys. 34 (1993), no. 6, 2242—2256. 

[12] S.-J. Chang and K.-J. Shi, Billiard systems on quadric surfaces and the Poncelet theorem, J. 
Math. Phys. 30 (1989), no. 4, 798-804. 

[13] Gaston Darboux, Lecons sur la theorie generate des surfaces et les applications geometriques 
du calcul infinitesimal, Vol. 2 and 3, Gauthicr-Villars, Paris, 1914. 

[14] A. Delshams, Y. Fedorov, and R. Rami'rez-Ros, Homoclinic billiard orbits inside symmetri- 
cally perturbed ellipsoids, Nonlinearity 14 (2001), 1141-1195. 

[15] Vladimir Dragovic and Milena Radnovic, Geometry of integrable billiards and pencils of 
quadrics, Journal Math. Purcs Appl. 85 (2006), 758-790. 

[16] , Hyperelliptic Jacobians as Billiard Algebra of Pencils of Quadrics: Beyond Poncelet 

Porisms, Adv. Math. 219 (2008), no. 5, 1577-1607. 

[17] Philip Griffiths and Joe Harris, A Poncelet theorem in space, Comment. Math. Helvetici 52 
(1977), no. 2, 145-160. 

[18] , On Cayley 's explicit solution to Poncelet's porism, Enseign. Math. 24 (1978), no. 1-2, 

31-40. 



18 



VLADIMIR DRAGOVIC AND MILENA RADNOVIC 



[19] Carl Jacobi, Vorlesungen iiber Dynamic. Gesammelte Werke, Supplementband, Berlin, 1884. 
[20] Horst Knorrcr, Geodesies on the ellipsoid, Invcntiones Math. 59 (1980), 119-143. 

[21] , Singular fibres of the momentum mapping for integrable Hamiltonian systems, J. 

Reine Angew. Math. 355 (1985), 67-107. 
[22] Valcry Kozlov and Dmitry Treshchev, Billiards, Amer. Math. Soc, Providence RI, 1991. 
[23] Henri Lebesgue, Les coniques, Gauthier-Villars, Paris, 1942. 

[24] Jiirgen Moser and Alexander Veselov, Discrete versions of some classical integrable systems 

and factorization of matrix polynomials, Comm. Math. Phys. 139 (1991), no. 2, 217-243. 
[25] Jiirgen Moser, Geometry of quadrics and spectral theory, The Chern Symposium, Springer, 

New York-Berlin, 1980, pp. 147-188. 
[26] Jean Victor Poncelet, Traite des proprietes projectives des figures, Mett, Paris, 1822. 
[27] E. Previato, Some integrable billiards, SPT2002: Symmetry and Perturbation Theory (S. 

Abenda, G. Gaeta, and S. Walcher, eds.), World Scientific, Singapore, 2002, pp. 181-195. 
[28] Milcna Radnovic and Vcrcd Rom-Kcdar, Foliations of isonergy surfaces and singularities of 

curves, Regular and Chaotic Dynamics 13 (2008), no. 6, 645—668. 
[29] E. Shlizerman and V. Rom-Kcdar, Hierarchy of bifurcations in the truncated and forced 

nonlinear Schrddinger model, Chaos 15 (2005), no. 1. 
[30] D. Turaev and V. Rom-Kcdar, Soft billiards with corners, Journal of Statistical Physics 112 

(2003), no. 3/4, 765-813. 
[31] H. Waalkcns and H. R. Dullin, Quantum Monodromy in Prolate Ellipsoidal Billiards, Annals 

of Physics 295 (2002), no. 1, 81-112. 

Mathematical Institute SANU, Kneza Mihaila 36, Belgrade, Serbia 
Mathematical Physics Group, University of Lisbon, Portugal 
E-mail address: vladad@mi.sanu.ac.rs 

Mathematical Institute SANU, Kneza Mihaila 36, Belgrade, Serbia 
E-mail address: milena@mi.sanu.ac.rs 



